The majority of the literature in learning in MDPs studies stationary environments, where the underlying unknown We study episodic reinforcement learning under transition function and reward function are fixed. The rewards unknown adversarial corruptions in both and the next states are independently and identically the rewards and the transition probabilities of distributed given the current state and the learner's chosen the underlying system. We propose new algorithms action. Under this setting, the goal is to minimize which, compared to the existing results the regret, which is the difference between the learner's in (Lykouris et al., 2020), achieve strictly better cumulative rewards and the total rewards of the optimal regret bounds in terms of total corruptions for policy (Brafman & Tennenholtz, 2002; Azar et al., 2017; the tabular setting. To be specific, firstly, our Jin et al., 2018; Ok et al., 2018; Zanette & Brunskill, 2019; regret bounds depend on more precise numerical Simchowitz & Jamieson, 2019; Zhang et al., 2020). However, values of total rewards corruptions and transition these techniques are vulnerable to corruptions on the corruptions, instead of only on the total rewards or the transitions.